Dynamic Regret Minimization for Control of Non-stationary Linear Dynamical Systems

We consider the problem of controlling a Linear Quadratic Regulator (LQR) system over finite horizon $T$ with fixed and known cost matrices $Q,R$, but unknown non-stationary dynamics $\{A_t, B_t\}$. The sequence can be arbitrary, total variation, $V_T$, assumed to $o(T)$ controller. Under assumption that stabilizing, potentially sub-optimal controllers is available for all $t$, we present an algorithm achieves optimal dynamic regret $\tilde{\mathcal{O}}\left(V_T^{2/5}T^{3/5}\right)$. With piece-wise constant dynamics, our $\tilde{\mathcal{O}}(\sqrt{ST})$ where $S$ number switches. crux adaptive non-stationarity detection strategy, which builds on approach recently developed contextual Multi-armed Bandit problems. also argue non-adaptive forgetting (e.g., restarting or using sliding window learning static size) may not LQR problem, even when size optimally tuned knowledge $V_T$. main technical challenge in analysis prove ordinary least squares (OLS) estimator has small bias parameter estimated non-stationary. Our highlights key motif driving spirit bandit linear feedback locally quadratic cost. This more universal than itself, therefore believe results should find wider application.